feat: add evaluation framework and CI/CD integration for csv-processor sample#1179
Merged
feat: add evaluation framework and CI/CD integration for csv-processor sample#1179
Conversation
github-actions
bot
added
test:uipath-langchain
Chibionos
force-pushed
the
feat/csv-processor-evaluations
branch
from
adf54bc to
dc2658c
Compare
ameyjain
reviewed
Jan 23, 2026
ameyjain
approved these changes
Jan 23, 2026
Chibionos
force-pushed
the
feat/csv-processor-evaluations
branch
3 times, most recently
from
64ac81c to
9e2d436
Compare
…r sample Add comprehensive evaluation framework for testing file input scenarios in the csv-processor sample. ## Changes ### Sample Updates (samples/csv-processor/) - Add Output model to main.py for proper evaluation data access - Create test data (sales_data.csv, large_dataset.csv, minimal.csv) - Add custom evaluators: - CSVShapeEvaluator: validates CSV dimensions (rows × columns) - CSVColumnsEvaluator: verifies expected column names - AttachmentCreatedEvaluator: checks output attachment creation - Add evaluation set (file-input-tests-local.json) with 3 test cases - Update uipath.json with Output model schema ### CI/CD Integration (testcases/csv-processor-evals/) - Add testcase directory structure following existing patterns - Create run.sh script for automated evaluation runs - Add assert.py for validation of evaluation results - Configure pyproject.toml and uipath.json for testcase execution ### Core Fixes - Fix evaluator path resolution bug in _evaluator_factory.py - Support co-located .py and .json files in evaluators directory - Check direct path before falling back to /custom/ subdirectory - Add pandas dev dependency for CSV processing ## Evaluation Results All 3 test cases pass with perfect scores (1.0/1.0): - Test Sales Data CSV Processing - Test Large Dataset CSV Processing - Test Minimal CSV Processing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixed three critical issues causing evaluation tests to fail: 1. Added pandas dependency to testcases/csv-processor-evals/pyproject.toml - Agent requires pandas but it was missing from eval dependencies 2. Fixed file paths in file-input-tests-local.json - Updated paths to be relative to testcases directory - Changed from "test-data/..." to "../../samples/csv-processor/test-data/..." 3. Optimized UiPath initialization in main.py - Moved UiPath() initialization to platform mode only - Prevents authentication errors when testing with local files All tests now pass with scores of 1.0 for both evaluators. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Pandas was incorrectly added to the root project's dev dependencies when it should only be a dependency of: - samples/csv-processor (has pandas>=2.3.3) - testcases/csv-processor-evals (has pandas>=2.3.3) The csv-processor evaluations run in the testcase environment which already has pandas installed via its own pyproject.toml. The root UiPath SDK does not need pandas as a core dependency. Changes: - Remove pandas>=3.0.0 from root pyproject.toml dev dependencies - Restore uv.lock to pre-pandas state (removes pandas, numpy, and resolution-markers that were added together) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Chibionos
force-pushed
the
feat/csv-processor-evaluations
branch
from
9e2d436 to
04d43ea
Compare
radu-mocanu
reviewed
Jan 28, 2026
| "$id": "entry-points.json", | ||
| "entryPoints": [ | ||
| { | ||
| "filePath": "main", |
Collaborator
There was a problem hiding this comment.
why did we change the entrypoints schema here? this was our go-to sample for attachments usage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add comprehensive evaluation framework for testing file input scenarios in the csv-processor sample.
Changes
Sample Updates (
samples/csv-processor/)Outputmodel to main.py for proper evaluation data accessfile-input-tests-local.json) with 3 test casesuipath.jsonwith Output model schemaCI/CD Integration (
testcases/csv-processor-evals/)run.shscript for automated evaluation runsassert.pyfor validation of evaluation resultspyproject.tomlanduipath.jsonfor testcase executionCore Fixes
_evaluator_factory.py.pyand.jsonfiles in evaluators directory/custom/subdirectoryEvaluation Results
All 3 test cases pass with perfect scores (1.0/1.0):
Test plan
uv run uipath eval main evaluations/eval-sets/file-input-tests-local.json./testcases/csv-processor-evals/run.sh🤖 Generated with Claude Code